A computer system may include a Peripheral Component Interconnect (PCI) Express (PCIe) host bridge able to connect between a processor and a memory unit, a graphics card, or other units. PCIe is an Input/Output (I/O) protocol allowing transfer of packetized data over high-speed serial interconnects with flow control-based link management.
PCIe includes a Flow Control (FC) mechanism which utilizes header FC credits and data FC credits. For example, a first PCIe endpoint or device (“sending device”) intends to send 37 PCIe packets to a second PCIe endpoint or device (“receiving device”). The receiving device advertises its receive buffer capability, by allocating a number of FC credits (for example, 20 FC credits) for incoming PCIe packets expected to be received from the sending device, and communicating the FC credit allocation to the sending device.
The sending device utilizes a parameter to track the number of FC credits allocated by the receiving device and remaining available to the sending device. Based on the advertised FC credit allocation information, the sending device sends 20 PCIe packets to the receiving device. Since the transmission consumes 20 FC credits, the sending device decreases 20 FC credits from the value of the parameter that tracks the number of FC credits available to the sending device.
The receiving device periodically advertises updates to the number of available receive buffer FC credits, for example, when receive buffer space becomes available. This is performed by issuing an FC Update (FCU) from the receiving device to the sending device. For example, upon reception and processing of PCIe packets corresponding to 15 out of the 20 FC credits, the receiving device advertises that additional 15 FC credits are available for subsequent send operations of the sending device.
The sending device has a PCIe packet pending for transmission, which consumes 17 FC credits. However, the receiving device advertises a total of 15 FC credits available for new PCIe packets incoming from the sending device. In accordance with the PCIe protocol, the sending device is not allowed to initiate PCIe packet transmission to the receiving device until all the FC credits, which are required in order to accept and store the PCIe packet(s) by the receiving device, are obtained by the sending device. In this situation, the sending device suspends packet transmission until two additional FC credits are obtained. This may result in delays in initiating PCIe transmissions, as well as reduced throughput and non-optimal utilization of the PCIe link.
In some systems, transmission delays may depend on the size of the receive buffer of the receiving device, and/or on the frequency of the FCU cycle. A relatively small receive buffer may become available rapidly; but may also result in frequent pauses in data flow, due to a substantially constant lack of FC credits, thereby degrading the effective bandwidth.
Furthermore, significant FC processing and/or update latency may require additional receive buffer space, in order to compensate for FCU delays. In a fast PCIe link, FCU delays may be equivalent to delays in transmission of large amounts of data. Therefore, a significant portion of the receive buffer may be required to compensate for FCU latency, and the effective utilization of the receive buffer may be low.